Papers with cascaded framework
Towards Multilingual spoken Visual Question Answering system using Cross-Attention (2025.coling-main)
Copied to clipboard
| Challenge: | Visual question answering (VQA) is a multi-modal translation challenge that requires the analysis of both images and questions simultaneously to generate appropriate responses. |
| Approach: | They propose a textless multilingual visual question answering dataset that incorporates speech-based questions in English, german, spanish and french. |
| Outcome: | The proposed framework is superior to existing frameworks for speech-based VQA . the proposed framework can generate better results for image, text and audio representations . |
RT-VQ2A2: Real Time Vector Quantized Question Answering with ASR (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing frameworks for QA with large language models are difficult to implement due to noise, limited context length and latency. |
| Approach: | They propose a model-agnostic framework to address problems in QA with large language models. |
| Outcome: | The proposed framework reduces noise in the ASR output and the limited context length of LLMs and improves performance on the widely used Spoken-SQuAD dataset. |